Feb 26
25
Anthropic updates its crawler documentation explaining what its crawlers do, and what blocking them will result in. Anthropic has three primary bots and they include ClaudeBot, Claude-User and Claude-SearchBot.
View post:
Anthropic Updates Crawler Docs: ClaudeBot, Claude-User & SearchBot